Decorrelation transform for a color filter array pattern image

ABSTRACT

A concept for coding a CFA pattern image is provided, wherein, for each of the sample clusters of the CFA pattern image, having first to fourth sample positions, first to fourth color coordinates are computed, so as to provide an image representation of the CFA pattern image.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of copending InternationalApplication No. PCT/EP2020/060790, filed Apr. 15, 2020, which isincorporated herein by reference in its entirety.

TECHNICAL FIELD

Embodiments of the present disclosure relate to an apparatus for codinga CFA pattern image into an image representation and to an apparatus fordecoding an image representation into a CFA pattern image. Embodimentsof the present disclosure also relate to a transformation for CFApattern images, and in particular to a color decorrelation or a combinedcolor and spatial decorrelation for CFA pattern images.

BACKGROUND OF THE INVENTION

Typical digital cameras are equipped with so-called CFA pattern sensors(for example Bayer sensors) that consist of a regular grid of sensorelements, overlaid with a regular pattern (or array) of color filters togenerate three color channels (see FIG. 1 ). Typically, filterscorrespond to red, green and blue light, but other filter arrangementsare also possible, e.g. red, clear (no filter), blue. This way, threeindependent color channels can be captured, though they are not alignedto each other and additional post-processing is performed to generate afull color image from the captured image.

In order to save bandwidth and processing power near the sensor, it isthe intent to compress the CFA sensor data as-is, i.e. withoutreconstruction (or post-processing) to a full-resolution RGB(true-color) image, transmit the compressed CFA image signal, andreconstruct it at the receiver side. The generation of a full-resolutiontrue-color image, i.e. the postprocessing step, is then performed at thedecoder side on the reconstructed CFA image data. As this entails thetransmission of only 1 component per sensor instead of 3, bandwidth isalready reduced.

Effective transmission and bandwidth reduction usually further includeslossy compression of the CFA data, which consists of multiple steps;First, a decorrelation across components, denoted the “colordecorrelation”, a spatial signal decorrelation—e.g. by a wavelet or DCT(discrete cosine transformation)—followed by quantization (irreversiblereduction of precision), followed by entropy coding of the data.

Both the color and spatial decorrelation processes aim for facilitatingand efficient encoding process and a high compression rate combined witha high quality of the decoded image.

SUMMARY

An embodiment may have an apparatus for decoding an image representationinto a color filter array (CFA) pattern image, wherein the CFA patternimage has first samples of a first channel, second samples of a secondchannel, and third samples of a third channel, wherein the CFA patternimage is partitioned into sample clusters, each of which has first tofourth sample positions with one of the first samples located at thefirst sample position, one of the second samples located at the secondsample position, and two third samples located at the third and fourthsample positions, respectively, wherein the apparatus is configured toderive, for each sample cluster, a first color coordinate for the firstsample position of the respective sample cluster, a second colorcoordinate for the second sample position of the respective samplecluster, a third color coordinate for the third sample position of therespective sample cluster and a fourth color coordinate for the fourthsample position of the respective sample cluster, wherein the apparatusis configured to decode the image representation into the CFA patternimage by computing for the third sample position of each sample cluster,the third sample at the respective third sample position by forming adifference of the third color coordinate at the third sample positionand a first filtered color coordinate which is derived by forming athird weighted sum of first and second color coordinates located atfirst and second sample positions neighboring the third sample positionin a manner so that a sum of weights of the third weighted sum is largerthan one half, for the fourth sample position of each sample cluster,the third sample at the respective fourth sample position by forming adifference of the fourth color coordinate at the fourth sample positionand a second filtered color coordinate which is derived by forming afourth weighted sum of first and second color coordinates located atfirst and second sample positions neighboring the fourth sample positionin a manner so that a sum of weights of the fourth weighted sum islarger than one half, for the first sample position of each samplecluster, the first sample at the respective first sample position byforming a summation of the first color coordinate at the first sampleposition and a first filtered third sample value derived by forming afirst weighted sum of third samples computed for third and fourth samplepositions neighboring the first sample position, for the second sampleposition of each sample cluster, a second sample at the respectivesecond sample position by forming a summation of the second colorcoordinate at the second sample position and a second filtered thirdsample value derived by forming a second weighted sum of third samplescomputed for third and fourth sample positions neighboring the secondsample position.

Another embodiment may have an apparatus for coding a color filter array(CFA) pattern image into an image representation, wherein the CFApattern image has first samples of a first channel, second samples of asecond channel, and third samples of a third channel, wherein the CFApattern image is partitioned into sample clusters, each of which hasfirst to fourth sample positions with one of the first samples locatedat the first sample position. one of the second samples located at thesecond sample position, and two third samples located at the third andfourth sample positions, respectively, wherein the apparatus isconfigured to code the CFA pattern image into the image representationby computing for the first sample position of each sample cluster, afirst color coordinate so that the first color coordinate isrepresentative of a difference of the first sample at the first sampleposition and a first filtered third sample value derived by forming afirst weighted sum of third samples neighboring the first sampleposition, for the second sample position of each sample cluster, asecond color coordinate so that the second color coordinate isrepresentative of a difference of the second sample at the second sampleposition and a second filtered third sample value derived by forming asecond weighted sum of third samples neighboring the second sampleposition, for the third sample position of each sample cluster, a thirdcolor coordinate by forming a summation of the third sample at the thirdsample position and a first filtered color coordinate which is derivedby forming a third weighted sum of first and second color coordinatescomputed for first and second sample positions neighboring the thirdsample position in a manner so that a sum of weights of the thirdweighted sum is larger than one half, for the fourth sample position ofeach sample cluster, a fourth color coordinate by forming a summation ofthe third sample at the fourth sample position and a second filteredcolor coordinate which is derived by a fourth weighted sum of first andsecond color coordinates computed for first and second sample positionsneighboring the fourth sample position in a manner so that a sum ofweights of the fourth weighted sum is larger than one half.

Another embodiment may have a method for decoding an imagerepresentation into a CFA pattern image, wherein the CFA pattern imagehas first samples of a first channel, second samples of a secondchannel, and third samples of a third channel, wherein the CFA patternimage is partitioned into sample clusters, each of which has first tofourth sample positions with one of the first samples located at thefirst sample position, one of the second samples located at the secondsample position, and two third samples located at the third and fourthsample positions, respectively, wherein the method has deriving, foreach sample cluster, a first color coordinate for the first sampleposition of the respective sample cluster, a second color coordinate forthe second sample position of the respective sample cluster, a thirdcolor coordinate for the third sample position of the respective samplecluster and a fourth color coordinate for the fourth sample position ofthe respective sample cluster, wherein the method has decoding the imagerepresentation into the CFA pattern image by computing for the thirdsample position of each sample cluster, the third sample at therespective third sample position by forming a difference of the thirdcolor coordinate at the third sample position and a first filtered colorcoordinate which is derived by forming a third weighted sum of first andsecond color coordinates located at first and second sample positionsneighboring the third sample position in a manner so that a sum ofweights of the third weighted sum is larger than one half, for thefourth sample position of each sample cluster, the third sample at therespective fourth sample position by forming a difference of the fourthcolor coordinate at the fourth sample position and a second filteredcolor coordinate which is derived by forming a fourth weighted sum offirst and second color coordinates located at first and second samplepositions neighboring the fourth sample position in a manner so that asum of weights of the fourth weighted sum is larger than one half, forthe first sample position of each sample cluster, the first sample atthe respective first sample position by forming a summation of the firstcolor coordinate at the first sample position and a first filtered thirdsample value derived by forming a first weighted sum of third samplescomputed for third and fourth sample positions neighboring the firstsample position, for the second sample position of each sample cluster,a second sample at the respective second sample position by forming asummation of the second color coordinate at the second sample positionand a second filtered third sample value derived by forming a secondweighted sum of third samples computed for third and fourth samplepositions neighboring the second sample position.

Still another embodiment may have a method for coding a color filterarray (CFA) pattern image into an image representation, wherein the CFApattern image has first samples of a first channel, second samples of asecond channel, and third samples of a third channel, wherein the CFApattern image is partitioned into sample clusters, each of which hasfirst to fourth sample positions with one of the first samples locatedat the first sample position, one of the second samples located at thesecond sample position, and two third samples located at the third andfourth sample positions, respectively, wherein the method has coding theCFA pattern image into the image representation by computing for thefirst sample position of each sample cluster, a first color coordinateso that the first color coordinate is representative of a difference ofthe first sample at the first sample position and a first filtered thirdsample value derived by forming a first weighted sum of third samplesneighboring the first sample position, for the second sample position ofeach sample cluster, a second color coordinate so that the second colorcoordinate is representative of a difference of the second sample at thesecond sample position and a second filtered third sample value derivedby forming a second weighted sum of third samples neighboring the secondsample position, for the third sample position of each sample cluster, athird color coordinate by forming a summation of the third sample at thethird sample position and a first filtered color coordinate which isderived by forming a third weighted sum of first and second colorcoordinates computed for first and second sample positions neighboringthe third sample position in a manner so that a sum of weights of thethird weighted sum is larger than one half, for the fourth sampleposition of each sample cluster, a fourth color coordinate by forming asummation of the third sample at the fourth sample position and a secondfiltered color coordinate which is derived by a fourth weighted sum offirst and second color coordinates computed for first and second samplepositions neighboring the fourth sample position in a manner so that asum of weights of the fourth weighted sum is larger than one half.

Another embodiment may have a non-transitory digital storage mediumhaving stored thereon a computer program for performing the aboveinventive methods for decoding and coding, when said computer program isrun by a computer.

Another embodiment may have a data stream having the imagerepresentation generated with the above inventive apparatus forencoding.

Embodiments of the present disclosure according to a first aspect of thepresent invention are based on the finding that a CFA pattern image canbe more efficiently encoded if the energy of two channels of the CFApattern image is compacted into, or primarily make up, one outputcoordinate of the output image representation. In contrast to commontransformations, which aim for compacting the energy of three channelsof the CFA pattern image into one coordinate, the disclosed conceptenables a higher coding efficiency, for example in terms of an improvedsignal-to-noise ratio of the restored CFA pattern image.

Additionally, embodiments of the present disclosure according to asecond aspect of the present invention which may be combined with thefirst aspect are based on the finding, that the quality of coding a CFApattern image may be improved by considering, for the coding of a samplecluster such as, for example, a super pixel, such as for sake of spatialdecorrelation, samples of the CFA pattern image which are located in aportion of the CFA pattern image crossed by lines and rows of the CFApattern image including at least one line as well as at least one rowoffset to the samples of the sample cluster currently to be coded. Byconsidering a larger portion of the CFA pattern image, decorrelation maybe improved and, thus, the energy may be compacted more efficiently,allowing for a higher compression rate and are a lower signal-to-noiseratio.

According to the first aspect of the present disclosure, this finding isexploited for coding a CFA pattern image partitioned into sampleclusters having each one first sample of a first channel, one secondsample of a second channel. and two third samples of a third channel ofthe CFA pattern image. The coding of the CFA pattern image comprisescomputing, for the first and second sample positions of each samplecluster, a first and a second color coordinate, respectively, by forminga difference of the first sample or the second sample, respectively, ofthe respective sample cluster and a first or second filtered thirdsample value, respectively. A further step of the coding of the CFApattern image comprises computing, for the third and fourth sampleposition of each sample cluster, a third and fourth color coordinate,respectively, by forming a summation of the third sample or fourthsample of the respective sample cluster and a first or second filteredcolor coordinate, respectively. The first and second filtered colorcoordinates are derived by forming a third and a fourth weighted sum offirst and second color coordinates neighboring the respective third andfourth sample positions, wherein a sum of weights of each of the thirdand fourth weighted sums is larger than one half. It may be equal to, orlarger than, one. In an example, the sum is larger than one quarter forweights relating to the first color coordinates and larger than onequarter for weights relating to the second color coordinates. It may beequal to, or larger than, 0.5. Due to this choice of the weights of thethird and fourth weighted sums for the calculation of the third andfourth color coordinates, the first and second channels, in particularlow frequency components, are overweight in the third and fourth colorcoordinate with respect to the first channel or, alternatively speaking,the energy of the third and fourth color coordinates is primarilydetermined by the energy of the first and second channels of the CFApattern image, what turns out to end-up into a better decorrelation and,thereby, leads to a higher compression rate or a lower quantization lossin a following quantization and entropy coding process. In case of anRGB pattern with third channel being green and equal weights for thefirst and second samples, the third and fourth color coordinatesprimarily corresponds to a magenta color of the low pass filteredversion of the CFA image, or a mixture of blue and red, with no or onlya fraction of green at smaller energy.

According to embodiments, the weights of the third and fourth weightedsums consist of individual weights for weighting first color coordinatesand second color coordinates, so that the weights may be separatelyadapted to a sensitivity of the respective channel. By choosing theweights for the first and second color coordinates so that the first andthe second channel are equally weighted in the third and fourth colorcoordinate, the coding efficiency is increased.

According to embodiments, the weights for weighting first and secondcolor coordinates in the third and fourth weighted sums are eachimplemented as a power of two, wherein the exponent of each of thepowers of two is an integer number. Thus, a multiplication of first andsecond color coordinates with their respective weight may be performedas a bit shift operation which is computationally efficient.

According to embodiments, a set of operation modes is available for thecoding of the CFA pattern image, the first to third samples of whichbeing arranged in rows and columns. In the context of rows and columns,horizontal directions refer to direction with in a row of the CFApattern image, and a vertical directions refer to directions alongcolumns of the CFA pattern image. Further, the definition of the rowsand columns is interchangeable by rotating the CFA pattern image by 90°,so that the attribution of rows and columns is to be understood notlimiting. For some examples, the coding of the CFA pattern image may beperformed row by row, and a first row which is referred to as beinglocated above a second row is to be understood as to be coded before thesecond row. Accordingly, the second row is located below the first row.The set of operation modes includes one or more of an isotropicoperation mode, a vertical causal operation mode and a stripe operationmode, which differ in a portion of the CFA pattern image which isconsidered for calculating a color coordinate for a selected sampleposition of a selected sample cluster. In all of the named operationmodes, several samples located in the rows, within which the selectedsample cluster is located, are included in the computation of the colorcoordinate. In the isotropic operation mode, samples located in rowsabove and below the row, within which the selected sample cluster islocated (named the current row), are included in the computation of thecolor coordinate of the respective type of the selected sample position.By including additional rows to the current rows of the CFA patternimage in the calculation, low-frequency components and high-frequencycomponents of the CFA pattern image in a vertical direction may beseparated, enabling a better compacting of the energy of the CFA patternimage into one or more color coordinates. In contrast, using onlysamples within the current rows, may perform the compaction of energyprimarily in a horizontal direction. Thus including additional rowsabove and/or below the current rows, may provide for a more efficientdata compression and a lower signal-to-noise ratio. In the verticalcausal operation mode, samples located in rows above but not below thecurrent row are included in the computation of the color coordinate ofthe respective type of the selected sample position. As no sampleslocated in rows below the current row are considered, the coding may beperformed close after receiving data signaling the samples of thecurrent row, without waiting for data signaling the samples ofsubsequent rows. Still, by considering samples located in rows above thecurrent row, a large portion of the CFA pattern image is considered incalculating the color coordinate, such increasing the quality. Thus,this operation mode may provide for a combination of a high codingquantity and a fast coding. In the stripe operation mode, exclusivelysamples located within the current row are included in the computationof the color coordinate. Thus, a sample of a specific sample positionmay be overwritten or dropped at the moment of calculating a colorcoordinate for the specific sample position and less data referring torows above the current row needs to be stored, compared to the verticalcausal and the isotropic operation mode. Thus, the stripe operation modecombines the advantage of being fast of the vertical causal operationmode with low requirements for memory.

According to an embodiment, the coding of the CFA pattern image includesfurther steps of computing a differential color coordinate third andfourth color coordinates and of computing a combinational colorcoordinate based on fourth and differential color coordinates. Thesesteps provided for a particularly high compaction of energy of the CFApattern image into one color coordinate, namely the combinational colorcoordinate, thus allowing for a high compression of the imagerepresentation by an entropy encoding process.

According to a second aspect of the present disclosure, the aboveintroduced concepts are exploited for coding a CFA pattern image havingfirst samples of a first channel, second samples of a second channel,and third samples of a third channel of the CFA pattern image, the firstto third samples being arranged in an array with rows and columns.Coding the CFA pattern image comprises computing, for each of the firstsample positions at which the first samples are located, a first colorcoordinate by forming a difference of the first sample at the respectivefirst sample position and a first filtered third sample value which isderived by forming a first weighted sum of third samples neighboring thefirst sample position. The third samples for the third weighted sumcomprise a third sample located in a row neighboring the one or morerows within which the sample cluster of the respective first sampleposition is located, named current rows in the following. For example,the neighboring row may be a row above the current rows, and may havebeen previously coded, so that the row is already available to thecoding process. By including third samples of a row neighboring thecurrent rows in the computation of the third weighted sum one additionalrow is used, but the quality of the coding is enhanced as explainedabove with respect to the vertical causal and the isotropic operationmode.

According to an embodiment, a set of operation modes is available forthe coding a CFA pattern image, the third of operation modes comprisinga first operation mode, for example one of the above introduced verticalcausal and isotropic operation modes, and a stripe operation mode. Inthe stripe operation mode, the third samples of the first weighted sumconsist of samples located within the current rows. Thus, a selectionbetween operation modes is possible, the stripe operation mode providingfor a fast coding and below memory requirements, and the first operationmode providing for a higher quality for higher efficiency of the coding.

Embodiments of the present disclosure according to the first and thesecond aspect as described above, may be used for coding a CFA patternimage into an image representation, which may be provided as such or maybe provided in an encoded representation of the image representation,and for obtaining the CFA pattern image from the image representation orthe encoded representation, respectively, by decoding the imagerepresentation. Thus, features, functionalities and advantages describedwith respect to the coding of the CFA pattern image are equallyapplicable to the coding and the decoding process.

BRIEF DESCRIPTION OF THE DRAWINGS

Advantageous implementations of the present disclosure are the subjectof dependent claims and embodiments are described in more detail belowwith respect to the figures, among which:

FIG. 1 shows an apparatus for coding a CFA pattern image according to anembodiment;

FIG. 2 shows an apparatus for coding a CFA pattern image according to afurther embodiment;

FIG. 3 shows an example of a CFA pattern image;

FIG. 4 shows an apparatus for coding a CFA pattern image according to afurther embodiment;

FIG. 5 shows an apparatus for decoding an image representation into aCFA pattern image according to an embodiment;

FIG. 6 shows an apparatus for decoding an image representation into aCFA pattern image according to a further embodiment;

FIG. 7 shows an apparatus for decoding an image representation into aCFA pattern image according to a further embodiment;

FIG. 8 shows an example of a CFA pattern image; and

FIG. 9 shows an example of current rows for coding a CFA pattern imagein a stripe operation mode.

DETAILED DESCRIPTION OF THE INVENTION

In the following, embodiments are discussed in detail, however, itshould be appreciated that the embodiments provide many applicableconcepts that can be embodied in a wide variety of image and videoprocessing and coding. The specific embodiments discussed are merelyillustrative of specific ways to implement and use the present concept,and do not limit the scope of the embodiments. In the followingdescription, a plurality of details is set forth to provide a morethorough explanation of embodiments of the disclosure. However, it willbe apparent to one skilled in the art that other embodiments may bepracticed without these specific details. In other instances, well-knownstructures and devices are shown in form of a block diagram rather thanin detail in order to avoid obscuring examples described herein. Inaddition, features of the different embodiments described herein may becombined with each other, unless specifically noted otherwise.

In the following description of embodiments, the same or similarelements or elements that have the same functionality are provided withthe same reference sign or are identified with the same name, and arepeated description of elements provided with the same reference numberor being identified with the same name is typically omitted. Hence,descriptions provided for elements having the same or similar referencenumbers or being identified with the same names are mutuallyexchangeable or may be applied to one another in the differentembodiments.

Multiple color transformations may be applied for color decorrelationfor encoding a CFA pattern image. These transformations may also includespatial decorrelation.

One example is the RCTD transformation, which is a four-componentvariant of the ROT transformation applied to true-color images for thepurpose of lossy or lossless compression. It separates the CFA patternas seen in FIG. 8 into 2×2 “super pixels”, each of which consists of onered, one blue and two green values, and from them, forms within eachsuper pixel, four components as such:

${t:=\left\lfloor \frac{G_{1} + G_{2}}{2} \right\rfloor}{Y:=\left\lfloor \frac{R + {2t} + B}{4} \right\rfloor}{{C_{b}:=B} - t}{{C_{r}:=R} - t}{{\Delta:=G_{1}} - G_{2}}$

[x] represents a rounding to an integer number, e.g. a floor orround-down operation to the next lower integer number, or a cell orround-up operation to the next higher integer number. Thisrepresentation is applicable to formulas throughout this application.The quadruple (Y,C_(b),C_(r),Δ) is then input to further compressionsteps. It is easily seen that this transformation is exactly invertiblewithout loss.

Recently, the Belgium company intoPix proposed a variant of the abovetransformation that includes a spatial decorrelation step along with thecolor transformation. To this end, it separates the CFA image intohorizontal stripes, each of which is two pixels—or one super-pixel—high,and as wide as the entire CFA image. Within each stripe, the colortransformation is performed in four lifting steps, each of whichinvertible by itself.

The first step generates two chroma-coordinates from red and blue, andthe neighboring green samples, as shown in FIG. 9 .

To define the transformation, indicate the sample value left to thecurrent position by the sub-index l, right to the current position bythe sub-index r. Similar, the sub-index t indicates the position to thetop, and the sub-index b the sample position below the current sample.Then, the first lifting step subtracts a weighted sum of green samplevalues from the red and blue values, and thus forms C_(b) and C_(r)channels similar to the above, though from a weighted neighborhood ofgreen values:

${{C_{b}:=B} - \left\lfloor \frac{G_{l} + {2G_{t}} + G_{r}}{4} \right\rfloor}{{C_{r}:=R} - \left\lfloor \frac{G_{l} + {2G_{b}} + G_{r}}{4} \right\rfloor}$

At the left and right edges, the non-existing left green channel isreplaced by the right green channel, and at the right image edge, thenon-existing right green channel is replaced by the left sample value.

In the second lifting step, two luma channels Y₁ and Y₂ are formed fromC_(b), C_(r) and the green channels as follows:

${{Y_{1}:=G} + \left\lfloor \frac{C_{r,l} + {2C_{b,b}} + C_{r,r}}{8} \right\rfloor}{{Y_{2}:=G} + \left\lfloor \frac{C_{b,l} + {2C_{r,t}} + C_{b,r}}{8} \right\rfloor}$

Where the Y₁ samples are located on the top, and the Y₂ samples arelocated on the bottom row of the two-line pixel arrangement as indicatedin FIG. 2 .

The third lifting step computes now from Y₁ and Y₂ a luminancedifference channel Δ on the top row. Similar to the above, the sub-indexrt indicates the sample value to the diagonal right-top, lt to theleft-top, rb to the right-bottom and lb to the left-bottom. Thus:

${\Delta:=Y_{1}} - \left\lfloor \frac{Y_{2,{lb}} + Y_{2,{rb}}}{2} \right\rfloor$

The last lifting step, finally, generates the output luminance signal asthe average from Y₁ and Y₂ through the usage of the difference signal Δ:

${Y:=Y_{2}} + \left\lfloor \frac{\Delta_{lt} + \Delta_{rt}}{4} \right\rfloor$

The quadruple (Y,C_(b),C_(r),Δ) is then input to further compressionsteps, where Δ is optionally excluded from any spatial decorrelation asit consists already of a high-pass filtered difference signal.

Similar transforms, i.e. transformations that combine spatial and colordecorrelation, have been discussed in the literature before, e.g. by T.Suzuki: “Lossless compression of CFA-sampled images using YDgCoCgtransforms with CDF wavelets”, in the Proc. of Intl. Conf. on ImageProc. (ICIP), 2018. The transformation discussed there is, however,based on the YCgCo transformation first proposed by H. S. Malvar and G.J. Sullivan in “Progressive-to-lossless compression ofcolor-filter-array images using macropixel spectralspatialtransformation,” in Proc. of DCC'12, Snowbird, Utah, April 2012, pp.3-12. The Suzuki transform is also based on lifting, similar to the onedescribed by intoPix, though is not confined to two lines.

The above described intoPix transformation aims for compacting theenergy of the three channels into one luma channel, namely Y, and isfurther limited to horizontal stripes. The idea of the first aspectaccording to the present disclosure is, as introduced before, thecompaction of the energy of two of three channels. According to thesecond aspect of the present disclosure, the transformation includes atleast one row of samples exceeding the rows of the currently codedsample cluster.

The first part of the detailed description introduces a general conceptwhich is implemented by embodiments according to both aspects of thepresent disclosure. Subsequently, different embodiments according to thefirst and second aspect are specified with reference to the generalconcept.

FIG. 1 shows a schematic representation of an example of an apparatus100 for coding a CFA pattern image 10 into an image representation 70.The CFA pattern image 10 comprises first samples 12 of a first channel,second samples 14 of a second channel, and third samples 16 of a thirdchannel which include third samples 16A, 168. For example, the first tothird channels consist of a red, a blue and a green channel, or a red, awhite and a blue channel.

For example, the CFA image pattern 10 is obtained from a plurality ofsensor elements, each of the sensor elements being associated with oneof the first channel, the second channel, and the third channel, whereinsensor elements being associated with a common channel are configured todetect light within a common wavelength range, which is specific to thechannel. For example, the CFA pattern image may have a Bayer pattern,e.g. as shown in FIG. 8 .

The CFA pattern image 10 is partitioned into sample clusters 20, eachcomprising one of the first samples 12 located at a first sampleposition 22, one of the second samples 14 located at a second sampleposition 24, a third sample 16A located at a third sample position 26and a further third sample 168 located at a fourth sample position 28,respectively.

The apparatus 100 is configured to code the CFA pattern image into theimage representation 70 by computing first to fourth color coordinates110, 120, 130, 140 for the first to fourth sample positions 22, 24, 26,28 for each of the sample clusters 20. In a step 1, which may bereferred to as a first lifting step, the apparatus 100 computes, foreach of the sample clusters 20, for the first sample position 22′ of arespective sample cluster 20′, a first color coordinate 110′, forexample a first chroma difference channel coordinate, so that the firstcolor coordinate one understood’ the present a difference of the firstsample) and the first sample position 22′ of the respective samplecluster 20′ and a first filtered third sample value 112′. The firstfiltered third sample value 12′ is derived by forming a first weightedsum 114′ of third samples 16 neighboring the first sample position 22′.It should be noted, that samples neighboring a specific sample positionmay be part of the same sample cluster is the specific sample positionbut may also be part of another sample cluster. Further, the apparatus100 is configured to compute, for the second sample position 24′ of therespective sample cluster 20′, a second color coordinate 120′, forexample a second chroma difference channel coordinate, so that thesecond color coordinate 120′ is representative of a difference of thesecond sample 14′ and the second sample position 14′ of the respectivesample cluster 20′ and a second filtered third sample value 122′. Thesecond filtered third sample value 122′ is derived by forming a secondweighted sum of 124′ of third samples 16 neighboring the second sampleposition 24′.

In a subsequent step 2, which may be referred to as a second liftingstep, the apparatus 100 is configured to compute, for each of the sampleclusters 20, for the third sample position 26′ of the respective samplecluster 20′, a third color coordinate 130′ by forming a summation, forexample an equal weighted summation, of the third sample 16A′ at thethird sample position 26′ of the respective sample cluster 20′ and afirst filtered color coordinate 132′ which is derived by forming a thirdweighted sum of 134′ of first and second color coordinates 110, 120computed for first and second sample positions 22, 24 neighboring thethird sample position 26′. The apparatus 100 is configured to compute,for the fourth sample position 28′ of each sample cluster 20′, a fourthcolor coordinate 140′ by forming a summation, for example an equalweighted summation, of the third sample 16B′ at the fourth sampleposition 28′ of the respective sample cluster 20′ and a second filteredcolor coordinate 142′ which is derived by a fourth weighted sum 144′ offirst and second color coordinates 110, 120 computed for first andsecond sample positions 22, 24 neighboring the fourth sample position28′.

For example, the apparatus 100 may provide, for each of the sampleclusters 20, the respective first to fourth color coordinates 110, 120,130, 140 in the image representation 70. A representation of the imagerepresentation 70 based on the first to fourth color coordinates 110,120, 130, 140 may be referred to as transformed representation 149.

Alternatively, the apparatus 100 may be configured to use the third andfourth color coordinates 130, 140, as intermediate values for computing,for each of the sample clusters 20, a differential color coordinate 250and a combinational color coordinate 260. FIG. 2 shows a schematicrepresentation of another example of the apparatus 100 comprisingadditionally to the computation of the first to fourth color coordinates110, 120, 130, 140, two additional steps for computing, based on thethird and fourth color coordinates 130, 140, a differential colorcoordinate 250 and a combinational color coordinate 260. In this case,the apparatus 100 may provide, for each of the sample clusters 20, therespective first, second, differential, and combinational colorcoordinates 110, 120, 250, 260 in the image representation 70. Arepresentation of the image representation 70 based on the first,second, differential, and combinational color coordinates 110, 120, 250,260 may be referred to as compacted transformed representation 249.

Accordingly, in a third step 3, which may be referred to as a thirdlifting step, the apparatus 100 may compute, for each of sample clusters20, for the third sample position 26′ of the respective sample cluster20′, the differential color coordinate 250′ by forming a difference ofthe third color coordinate 130′ at the third sample position 26′ of therespective sample cluster 20′ and a filtered fourth color coordinate252′. The filtered fourth color coordinate 252′ is derived by forming afifth weighted sum 254′, e.g. an equal weighted sum, of fourth colorcoordinates 140 computed for fourth sample positions 28 neighboring thethird sample position 26′.

In a fourth step 4, which may be referred to as a fourth lifting step,the apparatus 100 may compute, for each of the sample clusters 20, forthe fourth sample position 28′ of the respective sample cluster 20′, acombinational color coordinate 260 by forming a summation of the fourthcolor coordinate 140′ at the fourth sample position 28′ of therespective sample cluster 20′ and a filtered differential colorcoordinate 262′. The filtered differential color coordinate 262′ isderived by forming a sixth weighted sum 264′, for example an equalweighted sum, of differential color coordinates 250 computed for thirdsample positions 26 neighboring the fourth sample position 28′.

According to embodiments, for example as illustrated with respect to theCFA image pattern in FIG. 1 , the first to fourth sample positions arearranged in an array with rows of a first type 52 and rows of a secondtype 54. The rows of the first type 52 comprise alternately arrangedfirst sample positions 22 and third sample positions 26, and the rows ofthe second type 54 comprise alternately arranged second sample positions24 and fourth sample positions 28. The rows of the first type 52 and thesecond type are alternately the arranged along columns of the array. Thefirst sample positions 22 of the first type of rows 52 and the secondsample positions 24 of the second type of rows 54 are located indifferent columns 62, 64 of the array.

Thus, in the described array arrangement, each of the first samplepositions 22 is neighbored by four third samples 16, of which two thirdsamples 16A are located at third sample positions 26 neighboring thefirst sample position 22 in horizontal directions and two third samples16B are located at fourth sample positions 28 neighboring the firstsample position 22 in vertical directions. In this context, horizontaldirections refer to top and bottom, and a vertical directions refer toleft and right. Equally, each of the second sample positions 24 isneighbored by four third samples 16, of which two third samples 16 arelocated at third sample positions 26 neighboring the second sampleposition 24 in vertical direction and two third samples 16 are locatedat fourth sample positions 28 neighboring the second sample position 24in horizontal directions. According to the described example of thearrangement of sample positions, each of the sample clusters 20 islocated within a pair of two neighboring rows of samples comprising onerow of the first type 52 and one row of the second type 54. It ispointed out, the view arrangements of the sample clusters 20 withrespect to the first type and rows 52 and the second type of rows 54 aswell as with respect to a first type of columns 62 and a second type ofcolumns 64 as shown in FIG. 1 is exemplary, that is upper and lowerpositions as well as left and right positions of the rows and columns,respectively, may be exchanged, which may result in four differentarrangements of the first to fourth sample positions within the sampleclusters 20.

For example, the apparatus 100 may perform the calculation of the colorcoordinates, that is the first to fourth color coordinates 110, 120,130, 140 and optionally the differential and combinational colorcoordinates 250, 260, in a row wise manner, that is row by row.Different processing schemes or operation modes may be possible,differing in a size of a portion of the CFA pattern image which servesas source for respective input values for the first to sixth weightedsums 114′, 124′, 134′, 144′, 254′, 264′.

For example, the apparatus 100 may store the first to fourth colorcoordinates and/or the differential and combinational color coordinatescomputed during the respective first to fourth lifting steps inrespective data buffers, so that a subsequent lifting step may make useof the stored color coordinates. Buffered color coordinates 110, 120,130, 140, 250, 260 or samples 12, 14, 16 may be released or overwritten,when not required for the computation of color coordinates of furthersample clusters. Thus, the amount of data that is stored in therespective data buffer, hence may depend on a number of neighboringsample positions considered in a lifting step making use of therespective data buffer.

FIG. 3 illustrates an example of a portion of the CFA pattern image 10comprising the sample cluster 20′, for which to compute colorcoordinates. The sample cluster 20′ is located in the rows of the firstand the second type 52′, 54′, which are referred to as the current rows56.

For example, the apparatus 100 may operate in a stripe operation mode,or in-line operation mode, in which only samples located at samplepositions within the current rows 56 may contribute to the first tosixth weighted sums 114′, 124′, 134′, 144′, 254′, 264′. Alternatively,the apparatus 100 may operate in a vertical causal operation mode, inwhich additionally samples located in one or more rows above the currentrows 56 may contribute to one or more of the first to sixth weightedsums 114′, 124′, 134′, 144′, 254′, 264′. In yet another tentative, theapparatus 100 may operate an isotropic operation mode, in whichadditionally to the vertical causal operation mode samples located inone or more rows below the current rows 56 may contribute to one or moreof the first to sixth weighted sums 114′, 124′, 134′, 144′, 254′, 264′.

It is pointed out, that the detailed description of samples contributingto the first to sixth weighted sums 114′, 124′, 134′, 144′, 254′, 264′is conducted for sample positions which are not positions at an edge ofthe CFA pattern image. For edge sample positions, a missing sampleposition may be replaced by the sample position which is opposite of themissing sample position in view of the sample position for which therespective weighted sum is to be calculated.

In the stripe operation mode, only samples which are located within thecurrent rows 56 are considered as input values for the first to sixthweighted sums 114′, 124′, 134′, 144′, 254′, 264′. For example, the thirdsamples 16 for the first weighted sum 114′ may consist of the thirdsamples 16 located at third sample positions 26-1, 26′ neighboring thefirst sample position 22′ of the respective sample cluster 20′ inhorizontal directions and the third sample 16 located at the fourthsample position 28′. Accordingly, the third samples 16 for the secondweighted sum 124′ may consist of the third samples 16 located at thefourth sample positions 28-1, 28′ neighboring the second sample position24′ of the respective sample cluster 20′ in horizontal directions andthe third sample 16 located at the third sample position 26′. The firstand second color coordinates 110, 120 for computing the fourth weightedsum 144′ may consist of the second color coordinates 120 located at thesecond sample positions 24′, 24-1 neighboring the fourth sample position28′ of the respective sample cluster 20′ in horizontal directions andthe first color coordinate 110 located at the first sample position 22′neighboring the third sample position 26′ in a first vertical direction,for example a top direction. Further, the first and second colorcoordinates 110, 120 for computing the third weighted sum 134′ mayconsist of the first color coordinates 110 located at the first samplepositions 22′, 22-1 neighboring the third sample position 26′ of therespective sample cluster 20′ in horizontal directions and the secondcolor coordinate 120 located at the second sample position 24′neighboring the third sample position 26′ in a second verticaldirection, for example a bottom direction.

The fourth color coordinates 140 and the differential color coordinates250 for computing the fifth weighted sum 254′ and the sixth weighted sum264′, respectively, neighbor the respective third sample position 26′and fourth sample position 28′ in diagonal directions of the array. Afirst and a second diagonal direction refer to diagonal directions whichhave vertical components pointing towards the first vertical directionor top direction. A third and a fourth diagonal direction refer todiagonal directions which have vertical components pointing towards thesecond vertical direction or button direction.

In the stripe operation mode, the fourth color coordinates 140 forcalculating the fifth weighted sum 254′ may consist of the fourth colorcoordinates 140 computed for the fourth sample positions 28′, 28-1neighboring the third sample position 26′ of the respective samplecluster 20 in the third and the fourth diagonal direction of the array,for example the bottom left and button right diagonal directions.Accordingly, the differential color coordinates 250 of the sixthweighted sum 264′ may consist of the differential color coordinates 250computed for the third sample positions 26′, 26-1 neighboring the fourthsample position 28′ of the respective sample cluster 20′ and the firstand the second diagonal direction, for example the top left in the topright diagonal directions.

Because in the stripe operation mode samples may be used explicitly onlyfor calculating color coordinates of the current rows 56, a specificsample position may be replaced by the color coordinate calculated forthe specific sample position. Thus, this operation mode has particularlylow memory requirements. Still, it may improve the performance withrespect to state of the art transformation, for example by about 0.3 dBat low bitrates, due to the compaction of the energy of the first andthe second channel in one color coordinate of the image representation.

Another example of current rows 56 used for computing the respectivecolor coordinates is depicted in FIG. 9 for the case of the first, thesecond and the third channels being a red, a blue and a green channel.

A further possible operation mode is the vertical causal operation mode,which is explained with reference to FIG. 3 . In the vertical causaloperation mode, sample positions within one or more rows neighboring thecurrent rows 56 in a first vertical direction, for example the topdirection, may be included in the calculation of one or more of thefirst to sixth weighted sums 114′, 124′, 134′, 144′, 254′, 264′.Advantageously, the first vertical direction points towards rows of thearray which were received by the apparatus 100 temporally before thecurrent rows 56, Thus, the CFA pattern image 10 may be coded without anadditional temporal delay with respect to the stripe operation mode.

For example, in the vertical causal operation mode, the second weightedsum 124′, the fourth weighted sum 144′ and the sixth weighted sum 264′may be calculated as described with respect to the stripe operationmode. The third samples 16 for the first weighted sum 114′ may consistof the third samples 16 located at third sample positions 26-1, 26′neighboring the first sample position 22′ of the respective samplecluster 20′ in horizontal directions and the third samples 16 located atthe fourth sample positions 28′, 28-2 neighboring the first sampleposition 22′ in vertical directions. Additionally, the first and secondcolor coordinates 110, 120 for computing the third weighted sum 134′ mayconsist of the first color coordinates 110 located at the first samplepositions 22′, 22-1 neighboring the third sample position 26′ of therespective sample cluster 20′ in horizontal directions and the secondcolor coordinates 120 located at the second sample positions 24′, 24-2neighboring the third sample position 26′ in vertical directions. Thefourth color coordinates 140 for calculating the fifth weighted sum 254′may consist of the fourth color coordinates 140 computed for the fourthsample positions 28′, 28-1, 28-3, 28-4 neighboring the third sampleposition 26′ of the respective sample cluster 20 in the four diagonaldirections of the array. In other words, in examples of the verticalcausal mode, only top-neighbors are accessed, and thus no additionallatency is created, though one additional line must be buffered perlifting step. That is, the encoder, e.g. the apparatus 100, input bufferrequirements may be increased by four lines, and the decoder, e.g. theapparatus 500 of FIG. 5 , output buffer requirements may be increased byfour lines.

In the stripe operation mode and the vertical causal operation mode, thefirst to sixth weighted sums 114′, 124′, 134′, 144′ may be weighted suchthat a total weight of contributions from sample positions neighboringthe sample position for which the respective color coordinate iscalculated in horizontal directions equals a total weight ofcontributions from sample positions neighboring the sample position forwhich the respective color coordinate is calculated vertical direction.That is, in case only one vertically neighboring sample position isconsidered, the respective value may be weighted double.

A further possible operation mode is the isotropic operation mode, inwhich samples located in rows neighboring the current rows 56 in bothvertical directions are considered in the calculation of one or more orall of the color coordinates 110, 120, 130, 140, 250, 260. The firstweighted sum 114′, the third weighted sum 134′ and the fifth weightedsum 254′ may be calculated as described with respect to the verticalcausal operation mode. The third samples 16 for the second weighted sum124′ may comprise the third samples 16 located at the fourth samplepositions 28-1, 28′ neighboring the second sample position 24′ of therespective sample cluster 20′ in horizontal directions and the thirdsamples 16 located at the third sample positions 26′, 26-2 neighboringthe second sample position 24′ in vertical directions. In this case, oneadditional row below the current rows 56 with respect to the previouslydescribed operation modes is used for calculating the second colorcoordinates 120. Additionally but optionally, e.g. depending on amaximum acceptable temporal delay introduced by the consideration ofadditional rows located below the current rows 56, the first and secondcolor coordinates 110, 120 for computing the fourth weighted sum 144′may comprise the second color coordinates 120 located at the secondsample positions 24′, 24-1 neighboring the fourth sample position 28′ ofthe respective sample cluster 20′ in horizontal directions and the firstcolor coordinates 110 located at the first sample positions 22′, 22-2neighboring the third sample position 26′ in vertical directions. Inthis case, two additional rows below the current rows 56 with respect tothe previously described operation modes are used. Additionally butoptionally, the differential color coordinates 250 of the sixth weightedsum 264′ may consist of the differential color coordinates 250 computedfor the third sample positions 26′, 26-1, 26-2, 26-3 neighboring thefourth sample position 28′ of the respective sample cluster 20′ in thefour diagonal directions. In this case, four additional rows below thecurrent rows 56 with respect to the previously described operation modesare used. In other words, the encoder transformation delays its output70 by 4 lines or rows, and the decoder includes an additional delay by 4lines or rows, thus the end-to-end latency may be increased by 8 linesrows.

According to embodiments, the apparatus 100 comprises a set of operationmodes comprising one or more of the isotropic operation mode, thevertical causal operation mode and the stripe operation mode. Thus, oneof the set of operation modes may be selected depending on availablememory and depending on a maximum temporal delay which is acceptable forcoding and decoding the CFA pattern image 10.

According to embodiments, the apparatus 100 may select one of the set ofoperation modes for coding the CFA pattern image 10 and made signal theselected operation mode in a data stream, in which the imagerepresentation 70 is signaled.

According to embodiments, the first filtered third sample value 112′,the second filtered third sample value 122′, the first filtered colorcoordinate 132′, the second filtered color coordinate 142′, the filteredfourth color coordinate 252′, and the filtered differential colorcoordinate 262′ are obtained from the first weighted sum 114′, thesecond weighted sum 124′, the third weighted sum 134′, the fourthweighted sum 144′, the fifth weighted sum 254′, and the sixth weightedsum 264′ by a rounding operation, respectively, what may allow for anefficient computation and storage of the respective color coordinates.For example, the rounding operation may be a floor or a cell operation,which may round a number to the next lower or higher integer number,respectively.

According to embodiments, the first weighted sum 114′ and the secondweighted sum 124′ each represent an average of their respective thirdsamples. Thus, the first and second color values 110′, 120′ mayrepresent a difference between the first and second sample 12′, 14′,respectively, and a spatially filtered average value of the respectivethird samples neighboring the first and second sample positions 22′,24′.

According to embodiments, the weights of the third and fourth weightedsums 134′, 144′ consist of a first weight for weighting first colorcoordinates and a second weight for weighting second color coordinates.For example, the first weight may be implemented as a first power oftwo, and the second weight may be implemented as a second power of two,the exponents of the first and the second power of two being an integernumber, for example negative, zero, or positive.

In the following, examples for computing the color coordinates 110, 120,130, 140, 250, 260 are described in more detail. For illustrativepurposes, and in accordance with some embodiments it is assumed, thatthe first channel is a red channel, the second channel is a bluechannel, and the third channel is a green channel. Nevertheless, thedescribed transformation is also advantageous for differentconfigurations. For example, in further embodiments, the first and thesecond channel may be exchanged and/or the third channel may be a whitechannel. In the following formulas, R represents a first sample 12, Brepresents a second sample 22, and G represents a third sample 24 or, tobe more precise, the sample value thereof. Also for illustrativepurpose, but not limiting, the first vertical direction is chosen as topdirection, the second vertical direction is chosen as bottom direction,and the horizontal directions are chosen is left and right. Neighboringsamples of a sample position, for which the respective color coordinateis to be calculated, are indexed with l for left, r for right, t fortop, and b for bottom, and lt, lb, rt, rb for the corresponding diagonaldirections. Accordingly, the first color coordinate 110 may be referredto as a first chroma difference channel, C_(r), the second colorcoordinate 120 may be referred to as a second chroma difference channel,C_(b), the third color coordinate 130, may be referred to as a firstmagenta channel M₁, the fourth color coordinate 140 may be referred toas a second magenta channel M₂, the differential color coordinate 250may be referred to as a magenta differential channel Δ, and thecombinational color coordinate 260 may be referred to as average magentachannel M.

According to an embodiment, in each of the third and the fourth weightedsums 134′, 144′, a total of weights for first color coordinates 110equals a total of weights for second color coordinates 120, and the sumof weights of the respective weighted sum equals one. For thisparticular case, and in combination with the first and second weightedsums 114′, 124′ being equal weighted sums, this may form a spatialhigh-pass for the third channel, e.g. a green channel, and a highfrequency component or a DC component of the resulting combinationalcolor coordinate, contains mainly or even only components of the firstand second channel, e.g. red and blue components, but no component ofthe third channel, e.g. a green component. Thus, in case the first andsecond channels represent a rest and a blue channel, a combinationalcolor coordinate may be referred to as a magenta channel (M). In otherwords, the DC component of M may be the average of the red and bluechannel, coining its name “magenta”. Lifting, for example the third andthe fourth listing steps, however, may ensure that the green DCcomponent is reconstructable from C_(b), C_(r) and M. compacting theenergy of the CFA pattern image into a magenta channel may result a moreefficient coding, decreasing the signal-to-noise ratio, for example by0.3 dB.

For example, in the first listing step, the first color coordinate 110and the second color coordinate 120 may be computed according to thefollowing formulas, which are excellent very shown for the isotropicoperation mode:

${{C_{b}:=B} - \left\lfloor \frac{G_{l} + G_{t} + G_{b} + G_{r}}{4} \right\rfloor}{{C_{r}:=R} - \left\lfloor \frac{G_{l} + G_{t} + G_{b} + G_{r}}{4} \right\rfloor}$

In the “vertical causal” mode, G_(b) in the computation of C_(b) isreplaced by G_(t). In the “stripe operation mode, the previoussubstitution is made, and G_(l) in the computation of C_(r) is replacedby G_(b), too.

In the second lifting step, the third color coordinate 130 and a fourthcolor coordinate 140 may be computed, in the isotropic operation mode,according to:

${{M_{1}:=G_{1}} + \left\lfloor \frac{{2^{r}C_{r,l}} + {2^{b}C_{b,t}} + {2^{b}C_{b,b}} + {2^{r}C_{r,r}}}{8} \right\rfloor}{{M_{2}:=G_{2}} + \left\lfloor \frac{{2^{b}C_{b,l}} + {2^{r}C_{r,t}} + {2^{r}C_{r,b}} + {2^{b}C_{b,r}}}{8} \right\rfloor}$

For example, M₁ samples are located at the top row, and M₂ channels arelocated on the bottom row. In the vertical causal operation mode,C_(r,b) in the computation of M₂ may be replaced by C_(r,t), In thestripe operation mode, additionally C_(b,t) may be replaced by C_(b,b).The substitution rules are similar to that of the first lifting step,namely to create the unavailable samples by reflection of a similarsample from above or below.

In the above formulas for M1 and M2, weights for first color coordinates110 (C_(r)) are implemented as a first power of two, and weights forsecond color coordinates 120 (C_(b)) are implemented as a second powerof two. For the case r=b=1, the exponent of the first power of two andthe exponent of the second power of two are both equal to −2, resultingin a weight for each of the first and second color coordinates of ¼, sothat the sum of weights of the third and a fourth weighted sum 134′.144′, in this case, is one, thus providing an exemplary implementationfor compacting the energy of the CFA pattern image into a magentachannel, for the case of the first and the second channel being red andblue channels, respectively.

According to embodiments, the fifth weighted sum 254′ is an equalweighted average of the fourth color coordinates 140 of the fifthweighted sum 254′. For example, in this case, the differential colorcoordinates 250 may represent mainly differential components of thefirst two third channels, so that the differential color coordinates 250may be particularly small, which allows for an efficient encoding.

For example, the differential color coordinate 250 may be computed, inthe isotropic and the vertical causal operation mode, according to:

${\Delta:=M_{1}} - \left\lfloor \frac{M_{2,{lt}} + M_{2,{lb}} + M_{2,{rt}} + M_{2,{rb}}}{4} \right\rfloor$

In the stripe operation mode, M_(lt) may be replaced by M_(lb), andM_(rt) by M_(rb).

According to embodiments, the sixth weighted sum 264′ is an equalweighted average of the fourth color coordinates of the sixth weightedsum 264′ and a sum of weights of the sixth weighted sum 264′ equals onehalf.

For example, a combinational color coordinate 260 may be computedaccording to:

${M:=M_{2}} + \left\lfloor \frac{\Delta_{lt} + \Delta_{lb} + \Delta_{rt} + \Delta_{rb}}{8} \right\rfloor$

Thus, the described transformation may contact the energy of the firstand the second channel in the combinational color coordinate 216. In thevertical causal mode and the stripe operation mode, Δ_(lb) may bereplaced by Δ_(lt) and Δ_(rb) may be replaced by Δ_(rt.)

For example, the final output of the transformation, for example thecompacted transformed representation 249, may be an array of(M,C_(b),C_(r),Δ) samples. These may be input to further spatialdecorrelation, Optionally, Δ alone may not require additional spatialdecorrelation.

FIG. 4 illustrates another example of the apparatus 100 for coding theCFA pattern image 10. As shown, the apparatus 100 may additionallycomprise a quantizer 480 and an entropy coder 490. The quantizer 480 mayderive a quantized representation 482 by quantizing the transformedrepresentation 149, that is the first to fourth color coordinates 110,120, 130, 140 of the sample clusters 20, or, alternatively, thecompacted transformed representation 249, that is the first and secondcolor coordinates 110, 120 and the differential color coordinate 250 andthe combinational color coordinate 260 of the sample clusters 20. Theentropy coder 490 may encode the quantized representation 482, so as toobtain an encoded representation 495, which may be provided as the imagerepresentation 70.

Optionally, the apparatus 100 may further comprise a spatialdecorrelation stage 475 which may use a spatial decorrelation transform,for example a discrete cosine transformation or a discrete wavelettransformation to further spatially decorrelate the transformedrepresentation 149 or the compacted transformed representation 249. Inone example, the spatial decorrelation stage 475 applies the spatialdecorrelation transform to the first to fourth color coordinates 110,120, 130, 140, or the first, the second, the differential, and thecombinational color coordinates 110, 120, 250, 260 so as to obtain aspatially decorrelated representation 478, based on which the quantizer480 may derive the quantized representation 482. In another example, thespatial decorrelation stage 475 applies the spatial decorrelationtransform with the first, the second, and the combinational colorcoordinates 110, 120, 2060 so as to obtain a spatially decorrelatedrepresentation 478, and the quantizer 480 may derive the quantizedrepresentation 482 from the spatially decorrelated representation 478and the differential color coordinate 250 of the sample clusters 20.

FIG. 5 shows a schematic representation of an apparatus 500 for decodingthe image representation 70 so as to obtain the CFA pattern image 10.The apparatus 500 is configured to compute, for each sample cluster 20′of the sample clusters 20, the first sample 12′, the second sample 14′,and the two third samples 16A′, 16B′ from first to fourth colorcoordinates 110, 120, 130, 140.

The apparatus 500 may compute, for the third sample position 26′ of therespective sample cluster 20′, the third sample 16A′ at the respectivethird sample position 26′ by forming a difference of the third colorcoordinate 130′ at the third sample position 26′ of the respectivesample cluster 20′ and the first filtered color coordinate 132′.Further, the apparatus 500 may compute, for the fourth sample position28′ of each sample cluster 20′ the third sample 16B′ at the respectivefourth sample position 28′ by forming a difference of the fourth colorcoordinate 140′ at the fourth sample position 28′ of the respectivesample cluster 20′ and a second filtered color coordinate 142′. Thefirst and second filtered color coordinates 132, 142 may be obtainedfrom the first and second color coordinates as described with respect tothe apparatus 100.

In a subsequent step, the apparatus 500 may compute, for the firstsample position 22′ of each sample cluster 20′, the first sample 12′ atthe respective first sample position 22′ by forming a summation of thefirst color coordinate 110′ at the first sample position 22′ of therespective sample cluster 20′ and the first filtered third sample value112′. Further, the apparatus 500 may compute, for the second sampleposition 24′ of each sample cluster 20′, a second sample 14′ at therespective second sample position 14′ by forming a summation of thesecond color coordinate 120′ at the second sample position 24′ of therespective sample cluster 20′ and the second filtered third sample value122′. The first and second filtered third sample values 112, 122 may beobtained from third samples 16 computed for third and fourth samplepositions 26, 28 according to the previous step as described withrespect to the apparatus 100.

The apparatus 500 may comprise, equivalently to the apparatus 100, a setof operation modes for computing the first to sixth weighted sums.According to embodiments, the apparatus 500 may derive the operationmode to be used for decoding the image representation from a data streamsignaling the image representation and may select one of the set ofoperation modes accordingly.

FIG. 6 illustrates another example of the apparatus 500 configured toobtain the first to fourth sample 12, 14, 16A, 16B from first and secondcolor coordinates 110, 120 and differential and combinational colorcoordinates 250, 260, which may signal within the image representation70. The embodiment shown in FIG. 6 comprises two additional steps withrespect to the embodiment shown in FIG. 5 for computing the third andfourth color coordinates 130, 140 from the differential andcombinational color coordinates 250, 260.

For that purpose, the apparatus 500 may compute, for the fourth sampleposition 28′ of the respective sample cluster 20′, the fourth colorcoordinate 140′ at the respective fourth sample position 28′ by forminga difference of the combinational color coordinate 260′ at the fourthsample position 28′ of the respective sample cluster 20′ and thefiltered differential color coordinate 262′. The filtered differentialcolor coordinates 262 may be obtained from differential colorcoordinates 250 as described with respect to the apparatus 100.

In a subsequent step, the apparatus 500 may compute, for the thirdsample position 26′ of the respective sample cluster 20′, the thirdcolor coordinate 130′ at the respective third sample position 26′ byforming a summation of the differential color coordinate 250′ at thethird sample position 26′ of the respective sample cluster 20′ and afiltered fourth color coordinate 252′. The filtered fourth colorcoordinates 252 may be obtained from fourth color coordinates 140 asdescribed with respect to the apparatus 100.

FIG. 7 shows a schematic representation of another example of theapparatus 500, further comprising an entropy decoder 590 and adequantizer 580. The entropy decoder 590 may decode an encodedrepresentation 495, which may represent the image representation 70, soas to obtain the quantized representation 482. The dequantizer 580 mayuse an inverse quantization process so as to obtain, from the quantizedrepresentation 482, the first to fourth color coordinates 110, 120, 130,140, which may be indicated as the transformed representation 149, or,alternatively, the first, the second, the differential, and thecombinational color coordinates 110, 120, 250, 260, which may beindicated as the compacted transformed representation 249.

Optionally, the dequantizer 580 provides the spatially decorrelatedrepresentation 478 as introduced with respect to the apparatus 100 (cf.FIG. 4 ) and the apparatus 500 further comprises an inverse spatialdecorrelation stage 575 which may use an inverse spatial decorrelationtransform, for example an inverse discrete cosine transformation or aninverse discrete wavelet transformation, to derive the first to fourthcolor coordinates 110, 120, 130, 140, or the first, the second, thedifferential, and the combinational color coordinates 110, 120, 250,260.

According to another option, the dequantizer 580 provides the spatiallydecorrelated representation 478 and the differential color coordinates250 and the apparatus 500 further comprises the inverse spatialdecorrelation stage 575 which may use the inverse spatial decorrelationtransform to derive the first, the second, and the combinational colorcoordinates 110, 120, 260.

For further details of the apparatus 500 for decoding the imagerepresentation 70 into the CFA pattern image 10 reference is made to thedescription of the apparatus 100. Although, signals indicated withidentical reference sign may possibly differ, e.g. due to a quantizationloss, features of the apparatus 500 which are indicated with identicalreference signs as with respect to the apparatus 100 have equivalentfunctionality and may be implemented equivalently as introduced in therespective description.

In the following, embodiments of the present disclosure in accordancewith the first and second aspect are described. The embodiments arebased on the apparatus 100 is shown in, as described with respect toFIG. 1 .

In embodiments in accordance with the first aspect of the presentdisclosure, the third weighted sum 134′ is computed in a manner so thata sum of weights of the third weighted sum 134′ is larger than one half,and the fourth weighted sum 144′ is derived in a manner so that a sum ofweights of the fourth weighted sum 144′ is larger than one half.

In embodiments in accordance with the second aspect of the presentdisclosure, the third sample positions 26 of the third samples 16 of thefirst weighted sum 114′ exceed beyond the current rows 56 of the array,within which the respective sample cluster 20′ is located. For example,the first weighted sum 114′ may be calculated as described with respectto the isotropic operation mode or vertical causal operation mode.Optionally, also the computation of the second to sixth weighted sums124′, 134′, 144′, 254′, 264′ may be performed in accordance with one ofthe isotropic operation mode for the vertical causal operation mode. Inother words, in accordance with the second aspect of the presentdisclosure, the apparatus 100, 500 may comprise one of the isotropicoperation mode and the vertical causal operation mode. Differentpossible implementations of these operation modes are described above.

Further embodiments in accordance the second aspect, may additionallycomprise the stripe operation mode. For example, the apparatus 100, 500may select between the stripe operation mode and a first operation mode,which may, for example, correspond to the isotropic or the verticalcausal operation mode. According to embodiments, the apparatus 100 maybe configured to signal the operation mode used for computing the imagerepresentation 70 in a data stream, in which the image representation 70is signaled. Accordingly, the apparatus 500 may be configured to derive,from the data stream, the operation modes to be used for decoding theimage representation 70 so as to obtain the CFA pattern image.

Although some aspects have been described as features in the context ofan apparatus it is clear that such a description may also be regarded asa description of corresponding features of a method. Although someaspects have been described as features in the context of a method, itis clear that such a description may also be regarded as a descriptionof corresponding features concerning the functionality of an apparatus,

Some or all of the method steps may be executed by (or using) a hardwareapparatus, like for example, a microprocessor, a programmable computeror an electronic circuit. In some embodiments, one or more of the mostimportant method steps may be executed by such an apparatus.

The inventive encoded image signal can be stored on a digital storagemedium or can be transmitted on a transmission medium such as a wirelesstransmission medium or a wired transmission medium such as the Internet.

Depending on certain implementation requirements, embodiments of theinvention can be implemented in hardware or in software or at leastpartially in hardware or at least partially in software. Theimplementation can be performed using a digital storage medium, forexample a floppy disk, a DVD, a Blu-Ray, a CD, a ROM, a PROM, an EPROM,an EEPROM or a FLASH memory, having electronically readable controlsignals stored thereon, which cooperate (or are capable of cooperating)with a programmable computer system such that the respective method isperformed. Therefore, the digital storage medium may be computerreadable.

Some embodiments according to the invention comprise a data carrierhaving electronically readable control signals, which are capable ofcooperating with a programmable computer system, such that one of themethods described herein is performed.

Generally, embodiments of the present invention can be implemented as acomputer program product with a program code, the program code beingoperative for performing one of the methods when the computer programproduct runs on a computer. The program code may for example be storedon a machine readable carrier.

Other embodiments comprise the computer program for performing one ofthe methods described herein, stored on a machine readable carrier.

In other words, an embodiment of the inventive method is, therefore, acomputer program having a program code for performing one of the methodsdescribed herein, when the computer program runs on a computer.

A further embodiment of the inventive methods is, therefore, a datacarrier (or a digital storage medium, or a computer-readable medium)comprising, recorded thereon, the computer program for performing one ofthe methods described herein. The data carrier, the digital storagemedium or the recorded medium are typically tangible and/ornon-transitory.

A further embodiment of the inventive method is, therefore, a datastream or a sequence of signals representing the computer program forperforming one of the methods described herein. The data stream or thesequence of signals may for example be configured to be transferred viaa data communication connection, for example via the Internet.

A further embodiment comprises a processing means, for example acomputer, or a programmable logic device, configured to or adapted toperform one of the methods described herein.

A further embodiment comprises a computer having installed thereon thecomputer program for performing one of the methods described herein.

A further embodiment according to the invention comprises an apparatusor a system configured to transfer (for example, electronically oroptically) a computer program for performing one of the methodsdescribed herein to a receiver. The receiver may, for example, be acomputer, a mobile device, a memory device or the like. The apparatus orsystem may, for example, comprise a file server for transferring thecomputer program to the receiver.

In some embodiments, a programmable logic device (for example a fieldprogrammable gate array) may be used to perform some or all of thefunctionalities of the methods described herein. In some embodiments, afield programmable gate array may cooperate with a microprocessor inorder to perform one of the methods described herein. Generally, themethods may be performed by any hardware apparatus.

The apparatus described herein may be implemented using a hardwareapparatus, or using a computer, or using a combination of a hardwareapparatus and a computer.

The methods described herein may be performed using a hardwareapparatus, or using a computer, or using a combination of a hardwareapparatus and a computer.

In the foregoing Detailed Description, it can be seen that variousfeatures are grouped together in examples for the purpose ofstreamlining the disclosure. This method of disclosure is not to beinterpreted as reflecting an intention that the claimed examples requiremore features than are expressly recited in each claim. Rather, as thefollowing claims reflect, subject matter may lie in less than allfeatures of a single disclosed example. Thus the following claims arehereby incorporated into the Detailed Description, where each claim maystand on its own as a separate example. While each claim may stand onits own as a separate example, it is to be noted that, although adependent claim may refer in the claims to a specific combination withone or more other claims, other examples may also include a combinationof the dependent claim with the subject matter of each other dependentclaim or a combination of each feature with other dependent orindependent claims. Such combinations are proposed herein unless it isstated that a specific combination is not intended. Furthermore, it isintended to include also features of a claim to any other independentclaim even if this claim is not directly made dependent to theindependent claim.

While this invention has been described in terms of several embodiments,there are alterations, permutations, and equivalents which will beapparent to others skilled in the art and which fall within the scope ofthis invention. It should also be noted that there are many alternativeways of implementing the methods and compositions of the presentinvention. It is therefore intended that the following appended claimsbe interpreted as including all such alterations, permutations, andequivalents as fall within the true spirit and scope of the presentinvention.

1. An apparatus for decoding an image representation into a color filterarray (CFA) pattern image, wherein the CFA pattern image comprises firstsamples of a first channel, second samples of a second channel, andthird samples of a third channel, wherein the CFA pattern image ispartitioned into sample clusters, each of which comprises first tofourth sample positions with one of the first samples located at thefirst sample position, one of the second samples located at the secondsample position, and two third samples located at the third and fourthsample positions, respectively, wherein the apparatus is configured toderive, for each sample cluster, a first color coordinate for the firstsample position of the respective sample cluster, a second colorcoordinate for the second sample position of the respective samplecluster, a third color coordinate for the third sample position of therespective sample cluster and a fourth color coordinate for the fourthsample position of the respective sample cluster, wherein the apparatusis configured to decode the image representation into the CFA patternimage by computing for the third sample position of each sample cluster,the third sample at the respective third sample position by forming adifference of the third color coordinate at the third sample positionand a first filtered color coordinate which is derived by forming athird weighted sum of first and second color coordinates located atfirst and second sample positions neighboring the third sample positionin a manner so that a sum of weights of the third weighted sum is largerthan one half, for the fourth sample position of each sample cluster,the third sample at the respective fourth sample position by forming adifference of the fourth color coordinate at the fourth sample positionand a second filtered color coordinate which is derived by forming afourth weighted sum of first and second color coordinates located atfirst and second sample positions neighboring the fourth sample positionin a manner so that a sum of weights of the fourth weighted sum islarger than one half, for the first sample position of each samplecluster, the first sample at the respective first sample position byforming a summation of the first color coordinate at the first sampleposition and a first filtered third sample value derived by forming afirst weighted sum of third samples computed for third and fourth samplepositions neighboring the first sample position, for the second sampleposition of each sample cluster, a second sample at the respectivesecond sample position by forming a summation of the second colorcoordinate at the second sample position and a second filtered thirdsample value derived by forming a second weighted sum of third samplescomputed for third and fourth sample positions neighboring the secondsample position.
 2. The apparatus according to claim 1, wherein theweights of the third and fourth weighted sums comprise a first weightfor weighting first color coordinates and a second weight for weightingsecond color coordinates.
 3. The apparatus according to claim 2, whereinthe first weight is implemented as a first power of two, and wherein thesecond weight is implemented as a second power of two, wherein theexponents of the first and the second power of two is an integer number.4. The apparatus according to claim 1, wherein the first to fourthsample positions are arranged in an array with rows of a first type,each comprising alternately arranged first sample positions and thirdsample positions, and rows of a second type comprising alternatelyarranged second sample positions and third sample positions, wherein therows of the first type and rows of the second type are arrangedalternatingly along columns of the array, and wherein the third samplepositions within the rows of the first type on the one hand and thethird sample positions within the rows of the second type one the otherhand are located in different columns.
 5. The apparatus according toclaim 4, wherein the apparatus comprises a set of operation modescomprising at least one of an isotropic operation mode, a verticalcausal operation mode and a stripe operation mode, wherein, except forsample clusters located at an edge of the CFA pattern image, theapparatus is configured to exclusively include, for computing one ormore of the color coordinates for one of the sample positions of therespective sample cluster, in the isotropic operation mode, sampleslocated at sample positions within one or more current rows within whichthe respective sample cluster is located, and samples located in rowsabove and below the current rows, in the vertical causal operation mode,samples located at sample positions within the current rows, and sampleslocated in one or more rows above the current rows, and in the stripeoperation mode, samples located at sample positions within the currentrows,
 6. The apparatus according to claim 5, wherein the apparatus isconfigured to select one of the set of operation modes and wherein theapparatus is configured to derive the operation mode to be used fordecoding the image representation from a data stream signaling the imagerepresentation.
 7. The apparatus according to claim 1, wherein the thirdand fourth weighted sums are equal weighted averages of the respectivefirst and second color coordinates.
 8. The apparatus according to claim1, wherein the apparatus is configured to use a rounding operation toderive the first filtered third sample value from the first weightedsum, the second filtered third sample value from the second weightedsum, the first filtered color coordinate from the third weighted sum,and the second filtered color coordinate from the fourth weighted sum.9. The apparatus according to claim 1, wherein the apparatus isconfigured to derive the third and fourth color coordinates by computingfor the fourth sample position of each sample cluster, the fourth colorcoordinate at the respective fourth sample position by forming adifference of the combinational color coordinate at the fourth sampleposition and a filtered differential color coordinate which is derivedby forming a fifth weighted sum of differential color coordinateslocated at third sample positions neighboring the fourth sampleposition, for the third sample position of each sample cluster, thethird color coordinate at the respective third sample position byforming a summation of the differential color coordinate at the thirdsample position and a filtered fourth color coordinate which is derivedby forming a sixth weighted sum of fourth color coordinates computed forfourth sample positions neighboring the third sample position.
 10. Theapparatus according to claim 8, wherein the apparatus is configured touse a rounding operation to derive the filtered differential colorcoordinate from the fifth weighted sum, and the filtered fourth colorcoordinate from the sixth weighted sum.
 11. The apparatus according toclaim 9, wherein the first to fourth sample positions are arranged in anarray with rows of a first type, each comprising alternately arrangedfirst sample positions and third sample positions, and rows of a secondtype comprising alternately arranged second sample positions and thirdsample positions, wherein the rows of the first type and rows of thesecond type are arranged alternatingly along columns of the array, andwherein the third sample positions within the rows of the first type onthe one hand and the third sample positions within the rows of thesecond type one the other hand are located in different columns, andwherein the apparatus comprises an isotropic operation mode, wherein,the third samples of the first weighted sum comprise the third samplesneighboring the first sample position in vertical and horizontaldirections of the array, the third samples of the second weighted sumcomprise the third samples neighboring the second sample position invertical and horizontal directions of the array, the first and secondcolor coordinates of the third weighted sum comprise the first andsecond color coordinates computed for first and second sample positionsneighboring the third sample position in vertical and horizontaldirections of the array, the first and second color coordinates of thefourth weighted sum comprise the first and second color coordinatescomputed for first and second sample positions neighboring the fourthsample position in vertical and horizontal directions of the array, thefourth color coordinates of the fifth weighted sum comprise the fourthcolor coordinates computed for fourth sample positions neighboring thethird sample position in diagonal directions of the array, thedifferential color coordinates of the sixth weighted sum comprise thedifferential color coordinates computed for third sample positionsneighboring the fourth sample position in diagonal directions of thearray.
 12. The apparatus according to claim 9, wherein the first tofourth sample positions are arranged in an array with rows of a firsttype, each comprising alternately arranged first sample positions andthird sample positions, and rows of a second type comprising alternatelyarranged second sample positions and third sample positions, wherein therows of the first type and rows of the second type are arrangedalternatingly along columns of the array, and wherein the third samplepositions within the rows of the first type on the one hand and thethird sample positions within the rows of the second type one the otherhand are located in different columns, and wherein the apparatuscomprises a vertical causal operation mode, wherein, the third samplesof the first weighted sum comprise the third samples neighboring thefirst sample position in vertical and horizontal directions of thearray, the third samples of the second weighted sum comprise the thirdsamples neighboring the second sample position in horizontal directionsand a first vertical direction of the array, wherein a sum of weights ofthe second weighted sum for the third samples neighboring the secondsample position in horizontal directions equals a weight of the secondweighted sum for the third sample neighboring the second sample positionin the first vertical direction, the first and second color coordinatesof the third weighted sum comprise the first and second colorcoordinates computed for first and second sample positions neighboringthe third sample position in vertical and horizontal directions of thearray, the first and second color coordinates of the fourth weighted sumcomprise the first and second color coordinates computed for first andsecond sample positions neighboring the fourth sample position inhorizontal directions and the first vertical direction of the array, thefourth color coordinates of the fifth weighted sum comprise the fourthcolor coordinates computed for fourth sample positions neighboring thethird sample position in diagonal directions of the array, thedifferential color coordinates of the sixth weighted sum comprise thedifferential color coordinates computed for third sample positionsneighboring the fourth sample position in a first and a second diagonaldirection of the array, wherein vertical components of the first and thesecond diagonal directions point towards the same direction as the firstvertical direction.
 13. The apparatus according to claim 9, wherein thefirst to fourth sample positions are arranged in an array with rows of afirst type, each comprising alternately arranged first sample positionsand third sample positions, and rows of a second type comprisingalternately arranged second sample positions and third sample positions,wherein the rows of the first type and rows of the second type arearranged alternatingly along columns of the array, and wherein the thirdsample positions within the rows of the first type on the one hand andthe third sample positions within the rows of the second type one theother hand are located in different columns, and wherein the apparatuscomprises a stripe operation mode, wherein, the third samples of thefirst weighted sum comprise the third samples neighboring the firstsample position in horizontal directions and a second vertical directionof the array, wherein a sum of weights of the weighted sum for the thirdsamples neighboring the second sample position in horizontal directionsequals a weight of the weighted sum for the third sample neighboring thesecond sample position in the second vertical direction, the thirdsamples of the second weighted sum comprise the third samplesneighboring the second sample position in horizontal directions and afirst vertical direction of the array, wherein a sum of weights of theweighted sum for the third samples neighboring the second sampleposition in horizontal directions equals a weight of the weighted sumfor the third sample neighboring the first sample position in the firstvertical direction, the first and second color coordinates of the thirdweighted sum comprise the first and second color coordinates computedfor first and second sample positions neighboring the third sampleposition in horizontal directions and the second vertical direction ofthe array, the first and second color coordinates of the fourth weightedsum comprise the first and second color coordinates computed for firstand second sample positions neighboring the fourth sample position inhorizontal directions and the first vertical direction of the array, thefourth color coordinates of the fifth weighted sum comprise the fourthcolor coordinates computed for fourth sample positions neighboring thethird sample position in a third and a fourth diagonal direction of thearray, wherein vertical components of the third and the fourth diagonaldirections point towards the same direction as the second verticaldirection, the differential color coordinates of the sixth weighted sumcomprise the differential color coordinates computed for third samplepositions neighboring the fourth sample position in a first and a seconddiagonal direction of the array, wherein vertical components of thefirst and the second diagonal directions point towards the samedirection as the first vertical direction.
 14. The apparatus accordingto claim 9, wherein the sixth weighted sum is an equal weighted sum,wherein a sum of weights of the sixth weighted sum equals one half. 15.The apparatus according to claim 9, wherein the fifth weighted sum is anequal weighted average of the fourth color coordinates.
 16. Theapparatus according to claim 1, wherein the apparatus is configured toreceive an encoded representation as the image representation, andwherein the apparatus further comprises an entropy decoder configured todecode the encoded representation so as to derive a quantizedrepresentation of the image representation, and a dequantizer configuredto derive the first to fourth color coordinates, or the first, thesecond, the differential, and the combinational color coordinates by aninverse quantization process.
 17. The apparatus according to claim 1,wherein the apparatus is configured to receive an encoded representationas the image representation, and wherein the apparatus further comprisesan entropy decoder configured to decode the encoded representation so asto derive a quantized representation of the image representation, adequantizer configured to derive a spatially decorrelated representationbased on the quantized representation by an inverse quantizationprocess, and an inverse spatial decorrelation stage configured to derivethe first to fourth color coordinates, or the first, the second, thedifferential, and the combinational color coordinates from the spatiallydecorrelated representation by using an inverse spatial decorrelationtransform.
 18. The apparatus according to claim 9, wherein the apparatusis configured to receive an encoded representation as the imagerepresentation, and wherein the apparatus further comprises an entropydecoder configured to decode the encoded representation so as to derivea quantized representation of the image representation, a dequantizerconfigured to derive a spatially decorrelated representation and thedifferential color coordinates based on the quantized representation byan inverse quantization process, and an inverse spatial decorrelationstage configured to derive the first, the second, and the combinationalcolor coordinates from the spatially decorrelated representation byusing an inverse spatial decorrelation transform.
 19. An apparatus forcoding a color filter array (CFA) pattern image into an imagerepresentation, wherein the CFA pattern image comprises first samples ofa first channel, second samples of a second channel, and third samplesof a third channel, wherein the CFA pattern image is partitioned intosample clusters, each of which comprises first to fourth samplepositions with one of the first samples located at the first sampleposition, one of the second samples located at the second sampleposition, and two third samples located at the third and fourth samplepositions, respectively, wherein the apparatus is configured to code theCFA pattern image into the image representation by computing for thefirst sample position of each sample cluster, a first color coordinateso that the first color coordinate is representative of a difference ofthe first sample at the first sample position and a first filtered thirdsample value derived by forming a first weighted sum of third samplesneighboring the first sample position, for the second sample position ofeach sample cluster, a second color coordinate so that the second colorcoordinate is representative of a difference of the second sample at thesecond sample position and a second filtered third sample value derivedby forming a second weighted sum of third samples neighboring the secondsample position, for the third sample position of each sample cluster, athird color coordinate by forming a summation of the third sample at thethird sample position and a first filtered color coordinate which isderived by forming a third weighted sum of first and second colorcoordinates computed for first and second sample positions neighboringthe third sample position in a manner so that a sum of weights of thethird weighted sum is larger than one half, for the fourth sampleposition of each sample cluster, a fourth color coordinate by forming asummation of the third sample at the fourth sample position and a secondfiltered color coordinate which is derived by a fourth weighted sum offirst and second color coordinates computed for first and second samplepositions neighboring the fourth sample position in a manner so that asum of weights of the fourth weighted sum is larger than one half.
 20. Amethod for decoding an image representation into a CFA pattern image,wherein the CFA pattern image comprises first samples of a firstchannel, second samples of a second channel, and third samples of athird channel, wherein the CFA pattern image is partitioned into sampleclusters, each of which comprises first to fourth sample positions withone of the first samples located at the first sample position, one ofthe second samples located at the second sample position, and two thirdsamples located at the third and fourth sample positions, respectively,wherein the method comprises deriving, for each sample cluster, a firstcolor coordinate for the first sample position of the respective samplecluster, a second color coordinate for the second sample position of therespective sample cluster, a third color coordinate for the third sampleposition of the respective sample cluster and a fourth color coordinatefor the fourth sample position of the respective sample cluster, whereinthe method comprises decoding the image representation into the CFApattern image by computing for the third sample position of each samplecluster, the third sample at the respective third sample position byforming a difference of the third color coordinate at the third sampleposition and a first filtered color coordinate which is derived byforming a third weighted sum of first and second color coordinateslocated at first and second sample positions neighboring the thirdsample position in a manner so that a sum of weights of the thirdweighted sum is larger than one half, for the fourth sample position ofeach sample cluster, the third sample at the respective fourth sampleposition by forming a difference of the fourth color coordinate at thefourth sample position and a second filtered color coordinate which isderived by forming a fourth weighted sum of first and second colorcoordinates located at first and second sample positions neighboring thefourth sample position in a manner so that a sum of weights of thefourth weighted sum is larger than one half, for the first sampleposition of each sample cluster, the first sample at the respectivefirst sample position by forming a summation of the first colorcoordinate at the first sample position and a first filtered thirdsample value derived by forming a first weighted sum of third samplescomputed for third and fourth sample positions neighboring the firstsample position, for the second sample position of each sample cluster,a second sample at the respective second sample position by forming asummation of the second color coordinate at the second sample positionand a second filtered third sample value derived by forming a secondweighted sum of third samples computed for third and fourth samplepositions neighboring the second sample position.
 21. A method forcoding a color filter array (CFA) pattern image into an imagerepresentation, wherein the CFA pattern image comprises first samples ofa first channel, second samples of a second channel, and third samplesof a third channel, wherein the CFA pattern image is partitioned intosample clusters, each of which comprises first to fourth samplepositions with one of the first samples located at the first sampleposition, one of the second samples located at the second sampleposition, and two third samples located at the third and fourth samplepositions, respectively, wherein the method comprises coding the CFApattern image into the image representation by computing for the firstsample position of each sample cluster, a first color coordinate so thatthe first color coordinate is representative of a difference of thefirst sample at the first sample position and a first filtered thirdsample value derived by forming a first weighted sum of third samplesneighboring the first sample position, for the second sample position ofeach sample cluster, a second color coordinate so that the second colorcoordinate is representative of a difference of the second sample at thesecond sample position and a second filtered third sample value derivedby forming a second weighted sum of third samples neighboring the secondsample position, for the third sample position of each sample cluster, athird color coordinate by forming a summation of the third sample at thethird sample position and a first filtered color coordinate which isderived by forming a third weighted sum of first and second colorcoordinates computed for first and second sample positions neighboringthe third sample position in a manner so that a sum of weights of thethird weighted sum is larger than one half, for the fourth sampleposition of each sample cluster, a fourth color coordinate by forming asummation of the third sample at the fourth sample position and a secondfiltered color coordinate which is derived by a fourth weighted sum offirst and second color coordinates computed for first and second samplepositions neighboring the fourth sample position in a manner so that asum of weights of the fourth weighted sum is larger than one half.
 22. Anon-transitory digital storage medium having stored thereon a computerprogram for performing a method for decoding an image representationinto a CFA pattern image, wherein the CFA pattern image comprises firstsamples of a first channel, second samples of a second channel, andthird samples of a third channel, wherein the CFA pattern image ispartitioned into sample clusters, each of which comprises first tofourth sample positions with one of the first samples located at thefirst sample position, one of the second samples located at the secondsample position, and two third samples located at the third and fourthsample positions, respectively, wherein the method comprises deriving,for each sample cluster, a first color coordinate for the first sampleposition of the respective sample cluster, a second color coordinate forthe second sample position of the respective sample cluster, a thirdcolor coordinate for the third sample position of the respective samplecluster and a fourth color coordinate for the fourth sample position ofthe respective sample cluster, wherein the method comprises decoding theimage representation into the CFA pattern image by computing for thethird sample position of each sample cluster, the third sample at therespective third sample position by forming a difference of the thirdcolor coordinate at the third sample position and a first filtered colorcoordinate which is derived by forming a third weighted sum of first andsecond color coordinates located at first and second sample positionsneighboring the third sample position in a manner so that a sum ofweights of the third weighted sum is larger than one half, for thefourth sample position of each sample cluster, the third sample at therespective fourth sample position by forming a difference of the fourthcolor coordinate at the fourth sample position and a second filteredcolor coordinate which is derived by forming a fourth weighted sum offirst and second color coordinates located at first and second samplepositions neighboring the fourth sample position in a manner so that asum of weights of the fourth weighted sum is larger than one half, forthe first sample position of each sample cluster, the first sample atthe respective first sample position by forming a summation of the firstcolor coordinate at the first sample position and a first filtered thirdsample value derived by forming a first weighted sum of third samplescomputed for third and fourth sample positions neighboring the firstsample position, for the second sample position of each sample cluster,a second sample at the respective second sample position by forming asummation of the second color coordinate at the second sample positionand a second filtered third sample value derived by forming a secondweighted sum of third samples computed for third and fourth samplepositions neighboring the second sample position, when said computerprogram is run by a computer.
 23. A non-transitory digital storagemedium having stored thereon a computer program for performing a methodfor coding a color filter array (CFA) pattern image into an imagerepresentation, wherein the CFA pattern image comprises first samples ofa first channel, second samples of a second channel, and third samplesof a third channel, wherein the CFA pattern image is partitioned intosample clusters, each of which comprises first to fourth samplepositions with one of the first samples located at the first sampleposition, one of the second samples located at the second sampleposition, and two third samples located at the third and fourth samplepositions, respectively, wherein the method comprises coding the CFApattern image into the image representation by computing for the firstsample position of each sample cluster, a first color coordinate so thatthe first color coordinate is representative of a difference of thefirst sample at the first sample position and a first filtered thirdsample value derived by forming a first weighted sum of third samplesneighboring the first sample position, for the second sample position ofeach sample cluster, a second color coordinate so that the second colorcoordinate is representative of a difference of the second sample at thesecond sample position and a second filtered third sample value derivedby forming a second weighted sum of third samples neighboring the secondsample position, for the third sample position of each sample cluster, athird color coordinate by forming a summation of the third sample at thethird sample position and a first filtered color coordinate which isderived by forming a third weighted sum of first and second colorcoordinates computed for first and second sample positions neighboringthe third sample position in a manner so that a sum of weights of thethird weighted sum is larger than one half, for the fourth sampleposition of each sample cluster, a fourth color coordinate by forming asummation of the third sample at the fourth sample position and a secondfiltered color coordinate which is derived by a fourth weighted sum offirst and second color coordinates computed for first and second samplepositions neighboring the fourth sample position in a manner so that asum of weights of the fourth weighted sum is larger than one half, whensaid computer program is run by a computer.
 24. A data stream comprisingthe image representation generated with an apparatus according to claim19.